Analysis of the Minimal Residual Method Applied to Ill Posed Optimality Systems

We analyze the performance of the Minimal Residual Method applied to linear Karush-Kuhn-Tucker systems arising in connection with inverse problems. Such optimality systems typically have a saddle point structure and have unique solutions for all α > 0, where α is the parameter employed in the Tikhonov regularization. Unfortunately, the associated spectral condition number is very large for small values of α, which strongly indicates that their numerical treatment is difficult. Our main result shows that a broad range of linear ill posed optimality systems can be solved with a number of iterations of order O(ln(α−1)). More precisely, in the severely ill posed case the number of iterations needed by the Minimal Residual Method cannot grow faster than O(ln(α−1)) as α → 0. This result is obtained by carefully analyzing the spectrum of the associated saddle point operator: Except for a few isolated eigenvalues, the spectrum consists of bounded intervals. Krylov subspace methods handle such problems very well. We illuminate our theoretical findings with some numerical results for inverse problems involving partial differential equations. Our investigation is inspired by Prof. H. Egger’s discussion of similar results valid for the conjugate gradient algorithm applied to the normal equations.